Agenda

RStudio

Setup:
- How I do Python install and package management
- Setting up RStudio for Python use

Agenda

reticulate
- Environments
- Interacting with Python from R
- Type conversions
Live demo?

About Me

Who is Jennifer Knack?

Quick aside: RStudio

Disclaimer

R is my preferred coding language. All code in this talk, unless specified, is R code.

I use RStudio as my preferred IDE, and this talk was written from that perspective.

You can just as easily do this all in VSCode. But please…

Don’t use Jupyter Notebooks.

Hidden states hinder reproduciblity
- Cells can be run in any order, any number of times – according to one study¹, 73% of all Jupyter notebooks are not reproducible with straightforward approaches
Version control is nearly impossible because Jupyter notebooks are stored as single-line JSON files
- Merging branches with git versioning nearly always breaks because git version control works on a line-by-line basis
- Visualizing and understanding differences between versions is an impossible task
No linting, no style help, no IDE integration
Test-driven development is very difficult
Jupyter notebooks don’t scale well with big data

RStudio is objectively the best IDE

easy GUI with console and terminal windows
easy, integrated package management (for R)
easy, integrated report/presentation generation via Rmarkdown or Quarto
git integration
easily view environment objects and run history
linting, syntax help, and tab-completion all built in
easily navigate and view files and set working directory
plot view and help windows

“But why not just use VSCode?”

Because I don’t want to.

I like (and more importantly, know) RStudio.

R syntax used in this presentation

Because most (if not all) of you are primarily Python coders, I will introduce some R syntax I use regularly in this presentation.

Package loading

In R, one uses the library() function to bind packages to the environment. One then has access to all the functions and/or data in that package.

library(dplyr)

However, one can also access individual functions from a package using :: syntax.

dplyr::filter(df, col1 == "value")

This tells R which namespace to grab the function from without loading the whole package into the environment.

Throughout this talk, I will introduce new reticulate R functions in presentation text with :: syntax, but I will not use that syntax in code chunks since I will have already called library(reticulate).

Piping

Pipes are used to “pipe” the output from one function as the first input into another function. The original pipe in R was %>% from the magrittr package (also included in dplyr from the Tidyverse). For example:

library(dplyr)

df %>%
  filter(col1 == "value") %>%
  select(col2)

Starting in R v4.1.0 (2021), base R now has a native pipe |>, also called a “beak”, that works similarly to the magrittr %>%.

I prefer %>%. Ask Derek why he prefers |>.

Assignment

While one can use = for assignment to new objects, the <- is almost always used instead.

hi <- "Hello, world!"
print(hi)

[1] "Hello, world!"

Indexing

R uses base-1 indexing. More on this later.

Setup

Python install and package management

There are utilities to install and manage Python packages and environments in RStudio² but I don’t use them.

I use mamba³ instead.

Setting up RStudio to run Python

Tell RStudio where you installed Python

Go to Tools -> Global Options

Setting up RStudio to run Python

Enter the path to the python interpreter in your base environment

Setting up RStudio to run Python

Install the reticulate package

install.packages("reticulate")

reticulate⁴ is a Tidyverse package containing tools for interoperability between Python and R.

reticulate

What does reticulate do?⁵

Allows calling Python from R in multiple ways including from RMarkdown/Quarto, sourcing Python scripts, importing Python modules, and using Python interactively within an R session

Translates between R and Python objects (e.g., between R and Pandas data frames, or between R matrices and NumPy arrays)

Provides flexible binding to different versions of Python including virtual environments and Conda environments

A note from Posit about the philosophy behind Python tools in Rstudio

These tools are not intended for standalone Python work but rather explicitly aimed at the integration of Python into R projects (and as such are closely tied to the reticulate package).

They “strongly suggest” using one of the IDEs available for doing data science in Python for Python-only projects.⁶

What does reticulate do?

Allows calling Python from R in multiple ways including from RMarkdown/Quarto, sourcing Python scripts, importing Python modules, and using Python interactively within an R session
Translates between R and Python objects (e.g., between R and Pandas data frames, or between R matrices and NumPy arrays)

Provides flexible binding to different versions of Python including virtual environments and Conda environments

Set your environment

First thing you always want to do is set your environment:

library(reticulate)
use_condaenv("madpy")

Do this whether you’re working interactively, calling scripts, or authoring reports.

There is also reticulate::use_python() that allows you to specify an alternative version of python other than the one you set in your global options, or reticulate::use_virtualenv() to set a virtual environment instead of a conda environment.

What does reticulate do?

Allows calling Python from R in multiple ways including from RMarkdown/Quarto, sourcing Python scripts, importing Python modules, and using Python interactively within an R session

Translates between R and Python objects (e.g., between R and Pandas data frames, or between R matrices and NumPy arrays)
Provides flexible binding to different versions of Python including virtual environments and Conda environments

The `py` object

When you call library(reticulate), it creates the py object in the reticulate package environment.

It is the bridge between R and Python, through which you can run Python code and interact with Python objects.

The most common way you will interact with it is to access any Python object from the R environment using the $ operator, e.g., py$x.

Important

Always call library(reticulate) or you won’t be able to access the py object!

The `r` object

Similarly, reticulate creates the r object in the Python environment it creates. Through it, you can access R objects in the Python environment.

You can access R objects using the . operator, e.g., r.x.

Note

Examples of using the py and r objects are found further in this presentation

Importing modules

reticulate::import() can be used to import any installed Python module into your R environment.

os <- import("os")

Then you can call any function from that module in R using $.

os$listdir(".") |> head()

[1] ".git"                                ".gitignore"                         
[3] ".Rbuildignore"                       ".Rhistory"                          
[5] ".Rproj.user"                         "2024-04-24_python-in-rmarkdown.html"

If you’d like to access built in Python functions, use reticulate::import_builtins().

builtins <- import_builtins()
builtins$print('Hello, World!')

Hello, World!

Sourcing scripts

Let’s say I have a Python script that defines a function:

## this is Python code

def add(x,y):
  return x + y

If I’d like to use that function in R, I can source it using reticulate::source_python().

source_python('add.py')
add(5,10)

[1] 15

Executing code

Let’s say my collaborator wrote a Python script for processing some raw data. I’d like to work with the processed data in R, but my collaborator only provided me with the raw data and the script.

I know the script requires a variable file that is a character string pointing to the path of the raw data, and outputs a Pandas data frame called df.

I can use reticulate::py_run_string() and reticulate::py_run_file() to process the data, and then access any objects created into the Python main module using the py object exported by reticulate:

# Set the Python variable pointing to the raw data file
py_run_string("file = 'extdata/rawdata.csv'")

# run the processing script, which takes the file argument
py_run_file("process_raw_data.py")

# access the resulting df
py$df

Working in RMarkdown⁷

reticulate includes a Python engine for RMarkdown, and knitr v 1.18 (2017) and higher uses this engine by default.

Set your environment in your setup chunk:

Important

Always call library(reticulate) or you won’t be able to access the py object!

Then you can start inserting Python chunks just like you would R chunks, and knitr will knit everything together:

Just like when working interactively, you can access objects created in Python chunks in R by using the py object:

And you can access objects created in R chunks in Python by using the r object:

Working in Quarto

Quarto is an updated version of RMarkdown that is designed for a broader user base than just R users. I pretty much exclusively work in Quarto rather than RMarkdown these days.

Quarto provides all the support for Python that RMarkdown does, plus support for Jupyter.

Quarto supports rendering with the Jupyter kernel in addition to knitr and reticulate – just put jupyter: python3 in your YAML header and make sure the paths to Python and Jupyter are in your PATH:

---
title: "My Document"
jupyter: python3
---

You can also provide a full kernelspec in your YAML:

---
title: "My Document"
jupyter:
  kernelspec:
    name: xpython
    language: "python"
    display_name: "Python 3.7 (XPython)"
---

Because it can use the Jupyter kernel, Quarto CLI can render Jupyter notebooks too:

quarto render document.ipynb

There is other support for Python from the Quarto CLI as well, plus a VSCode Quarto plugin.⁸

Working with Python interactively

If you want to work with Python interactively, you can call reticulate::repl_python() to initiate a Python REPL embedded in the R console.

You can use the py and r objects to access objects between environments.

What does reticulate do?

Allows calling Python from R in multiple ways including from RMarkdown/Quarto, sourcing Python scripts, importing Python modules, and using Python interactively within an R session

Translates between R and Python objects (e.g., between R and Pandas data frames, or between R matrices and NumPy arrays)

Provides flexible binding to different versions of Python including virtual environments and Conda environments

Type conversions

When calling into Python, R data types are automatically converted to their equivalent Python types.

When values are returned from Python to R they are converted back to R types.⁹

The automatic conversion between R types and Python types works well in most cases, but sometimes you might want more control over the conversions.

Conversion table

R	Python	Examples in R
Single-element vector	Scalar	`1`, `1L`, `TRUE`, `"foo"`
Multi-element vector	List	`c(1.0, 2.0, 3.0)`, `c(1L, 2L, 3L)`
List of multiple types	Tuple	`list(1L, TRUE, "foo")`
Named list	Dict	`list(a = 1L, b = 2.0)`, `dict(x = x_data)`
Matrix/Array	NumPy ndarray	`matrix(c(1,2,3,4), nrow = 2, ncol = 2)`
Data Frame	Pandas Dataframe	`data.frame(x = c(1,2,3), y = c("a", "b", "c"))`
Function	Python function	`function(x) x + 1`
Raw	Python bytearray	`as.raw(c(1:10))`
NULL, TRUE, FALSE	None, True, False	`NULL`, `TRUE`, `FALSE`

Controling when conversion happens

If you’d like to work directly with Python objects by default you can pass convert = FALSE to the reticulate::import() function.

# import numpy and specify no automatic Python to R conversion
np <- import("numpy", convert = FALSE)

# do some array manipulations with NumPy
a <- np$array(c(1:4))
(sum_np <- a$cumsum())

array([ 1,  3,  6, 10])

# what is sum?
class(sum_np)

[1] "numpy.ndarray"         "python.builtin.object"

Then when you’re done working with the object in Python, you can convert it to an R object explicitly with reticulate::py_to_r() (and conversely, reticulate::r_to_py()).

# convert to R explicitly at the end and print object
(sum_r <- py_to_r(sum_np))

[1]  1  3  6 10

# what is sum_r?
class(sum_r)

[1] "array"

Note

We’ll be using both the sum_np and sum_r objects in examples later on, so take note of their difference

Defining the conversion

Numeric types are different between R and Python. For example, 42 in R is a float, while in Python it’s an integer.

If you want to explicitly define a number as an integer in R so that it’s passed as such to Python, use the L suffix:

class(42)

[1] "numeric"

class(42L)

[1] "integer"

If a Python API requires a list but you’re only passing it a single element, you can wrap it in list() from base R:

foo$bar(indexes = list(42L))

Similarly, if the Python API wants a tuple, you can use reticulate::tuple():

tuple("a", 5.5, FALSE)

And if the Python API wants a dictionary, you can use reticulate::dict():

dict(foo = "bar", index = 42L)

Indices

Python uses 0-based indices for collections:

sum_np[0L]

np.int64(1)

while R uses 1-based indices:

sum_r[1]

[1] 1

Note

Notice the need to explicitly use an integer when slicing the Python object

Python indices are non-inclusive for the end range, while R indices are:

sum_np[2L:4L]

array([ 6, 10])

sum_r[2:4]

[1]  3  6 10

And negative indexing in Python counts from the end of the container, while in R it removes that index:

sum_np[-1L]

np.int64(10)

sum_r[-1]

[1]  3  6 10

Arrays

Wait, do you mean vectors or matrices?

NumPy uses one type of object, the ndarray, for an indexed collection of numbers with any number of dimensions.

sum_np

array([ 1,  3,  6, 10])

class(sum_np)

[1] "numpy.ndarray"         "python.builtin.object"

R has multiple types of objects for this.

The R array is analogous to the NumPy ndarray and can also handle any number of dimensions.

sum_r

[1]  1  3  6 10

class(sum_r)

[1] "array"

More commonly, folks will use a numeric vector for 1-dimensional indexed collections of numbers.

rnorm(1:4)

[1]  1.3726301 -0.5999073 -1.0813687 -0.3859095

class(rnorm(1:4))

[1] "numeric"

These will be coerced to arrays by R automatically whenever they interact with other arrays mathematically.

A matrix is a subset of array with two dimensions.

matrix(rnorm(1:4),2,2)

           [,1]       [,2]
[1,] -2.0506431 -1.6153242
[2,] -0.2484595  0.9151527

class(matrix(rnorm(1:4),2,2))

[1] "matrix" "array"

The TL;DR for arrays¹⁰

R and Python represent 2+ -dimensional arrays in memory differently:

R only supports column-major order (FORTRAN-style)
NumPy supports both column-major and row-major (C-style) order, but defaults to row-major

WARNING

This can really mess everything up if you’re not careful!

Column- vs. Row-Major Order

Modified from image by Cmglee - Own work, CC BY-SA 4.0, https://commons.wikimedia.org/w/index.php?curid=65107030

Reshaping arrays

The R dim() function is used to reshape arrays in R. This works by changing the dim attribute of the array, effectively re-interpreting the array indices using column-major semantics.

# make an array from a vector of 4 elements:
u <- 1:4

# dim() uses column-major semantics
dim(u) <- c(2,2)
u

     [,1] [,2]
[1,]    1    3
[2,]    2    4

Remember though that NumPy uses row-major semantics by default; hence, its reshape method uses row-major semantics.

# NumPy reshape uses row-major semantics
np$reshape(np$arange(1,5), c(2L,2L))

array([[1., 2.],
       [3., 4.]])

So if you’re mixing R and Python code, you may get inconsistent results.

To overcome this, use reticulate::array_reshape() to reshape R arrays using row-major semantics.

# make an array from a vector
# of 4 elements:
u <- 1:4

# dim() uses
# column-major semantics
dim(u) <- c(2,2)
u

     [,1] [,2]
[1,]    1    3
[2,]    2    4

# array_reshape() uses
# row-major semantics
array_reshape(1:4, c(2,2))

     [,1] [,2]
[1,]    1    2
[2,]    3    4

Grouping while printing 3+D arrays

These are the exact same array:

(x <- np$arange(1, 9)$reshape(2L, 2L, 2L))

array([[[1., 2.],
        [3., 4.]],

       [[5., 6.],
        [7., 8.]]])

(y <- py_to_r(x))

, , 1

     [,1] [,2]
[1,]    1    3
[2,]    5    7

, , 2

     [,1] [,2]
[1,]    2    4
[2,]    6    8

So why do they look different?

Python groups by the first index when printing, while R groups by the last index:

x[0L,,]

array([[1., 2.],
       [3., 4.]])

x[,,0L]

array([[1., 3.],
       [5., 7.]])

y[1,,]

     [,1] [,2]
[1,]    1    2
[2,]    3    4

y[,,1]

     [,1] [,2]
[1,]    1    3
[2,]    5    7

This is analogous to slicing a cube along each of its axis planes:

Python slices at the first index:

array([[[1., 2.],
        [3., 4.]],

       [[5., 6.],
        [7., 8.]]])

R slices at the last index:

, , 1

     [,1] [,2]
[1,]    1    3
[2,]    5    7

, , 2

     [,1] [,2]
[1,]    2    4
[2,]    6    8

Note

The printout looks different from the cube because the cube is using row-major order, while the R array uses column-major order. The grouping is the same, though!

What about arrays from R to Python?

In the previous example I created an array in Python and ported it to R. What about the other way around?

(v <- array(1:8, c(2, 2, 2)))

, , 1

     [,1] [,2]
[1,]    1    3
[2,]    2    4

, , 2

     [,1] [,2]
[1,]    5    7
[2,]    6    8

(w <- np$array(v))

array([[[1, 5],
        [3, 7]],

       [[2, 6],
        [4, 8]]], dtype=int32)

The NumPy array will be created using column-major ordering:

w$flags

  C_CONTIGUOUS : False
  F_CONTIGUOUS : True
  OWNDATA : True
  WRITEABLE : True
  ALIGNED : True
  WRITEBACKIFCOPY : False

Remember:

F for “FORTRAN” (column-major order)
C for “C” (row-major order)

You can always create NumPy arrays in column-major order by passing the "F" flag:

np$reshape(np$arange(1, 9), c(2L, 2L, 2L))

array([[[1., 2.],
        [3., 4.]],

       [[5., 6.],
        [7., 8.]]])

np$reshape(np$arange(1, 9), c(2L, 2L, 2L), "F")

array([[[1., 5.],
        [3., 7.]],

       [[2., 6.],
        [4., 8.]]])

You can rearrange R arrays into row-major order:

matrix(1:4, 2, 2, byrow = TRUE)

     [,1] [,2]
[1,]    1    2
[2,]    3    4

However, it’s much more difficult with 3+ dimensions.

More array considerations

Dense R arrays are presented to Python as column-major NumPy arrays (FORTRAN-style)
All NumPy arrays (column-major, row-major, or otherwise) are presented to R as column-major arrays, since that’s all R can understand
R arrays are only copied to Python when they need to be, otherwise data are shared
NumPy arrays are always copied when moved into R arrays; this can sometimes lead to multiple copies of any one array in memory at one time

So in summary:

2+ -dimensional arrays are created in row-major order by default in NumPy and column-major order in R

3+ -dimensional arrays are printed by slicing by the first index in NumPy and by slicing by the last index in R

Therefore, I suggest you manipulate your arrays either in NumPy or in R to avoid catastrophe.

Sparse matrices

reticulate supports the conversion of sparse matrices created by the Matrix R package to and from SciPy CSC matrices.¹¹

I tried to make an example but working out the dependencies for scipy.sparse was way too much work.

https://rstudio.github.io/reticulate/articles/python_dependencies.html may have been helpful but I didn’t care enough.

Data Frames

The important points

As mentioned earlier, R data frames can be automatically converted to and from Pandas data frames. By default, columns are converted using the same rules governing R array <=> NumPy array conversion, with a couple extensions:

R factors <=> Python categorical variables
R POSIXt times <=> NumPy array with dtype=datetime64[ns]

The important points

If the R data frame has row names, the generated Pandas data frame will be re-indexed using those row names, and vice versa.

If a Pandas data frame has a DatetimeIndex, it is converted to character vectors as R only supports character row names.

Using Pandas nullable data types

Pandas out of the box handles NAs differently than R:

(df <- data.frame(
  int = c(NA, 1:4),
  num = c(NA, rnorm(4)),
  lgl = c(NA, rep(c(TRUE, FALSE), 2)),
  string = c(NA, letters[1:4])
))

  int        num   lgl string
1  NA         NA    NA   <NA>
2   1  0.5353032  TRUE      a
3   2  1.4232158 FALSE      b
4   3 -0.6032755  TRUE      c
5   4  0.5269580 FALSE      d

r_to_py(df)

          int       num    lgl string
0 -2147483648       NaN  False   None
1           1  0.535303   True      a
2           2  1.423216  False      b
3           3 -0.603275   True      c
4           4  0.526958  False      d

However, Pandas has experimental support for nullable data types (represented by pd.NA), but you have to enable it first:

# tell Pandas to use NAs
options(reticulate.pandas_use_nullable_dtypes = TRUE)

r_to_py(df)

    int       num    lgl string
0  <NA>      <NA>   <NA>   <NA>
1     1  0.535303   True      a
2     2  1.423216  False      b
3     3 -0.603275   True      c
4     4  0.526958  False      d

Final Points

For advanced Python users

For advanced Python users, there’s more documentation on

Contexts
Iterators
Functions
Creating high-level R interfaces for Python libraries

Check out https://rstudio.github.io/reticulate/articles/calling_python.html for deets.

Access to Python help from R

You can print documentation on any Python object using reticulate::py_help():

py_help(os$chdir)

This will open a text document outside of RStudio:

Help on built-in function chdir in module nt:

chdir(path)
    Change the current working directory to the specified path.

    path may always be specified as a string.
    On some platforms, path may also be specified as an open file descriptor.
      If this functionality is unavailable, using it raises an exception.

Some more reading

There is also this excellent article aimed at R users who are new to Python.

There is less available for the Python user who is new to R, but here’s a couple articles to get you started.

Your favorite search engine can tell you more too.

And I’m sure one of you will mention

rpy2, basically reticulate for the Python environment.

Documentation is not as robust as for reticulate, though.

Session info

R

sessionInfo()

R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 10 x64 (build 19045)

Matrix products: default


locale:
[1] LC_COLLATE=English_United States.utf8 
[2] LC_CTYPE=English_United States.utf8   
[3] LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C                          
[5] LC_TIME=English_United States.utf8    

time zone: America/Chicago
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.38.0

loaded via a namespace (and not attached):
 [1] digest_0.6.36     fastmap_1.2.0     xfun_0.46         Matrix_1.7-0     
 [5] lattice_0.22-6    rappdirs_0.3.3    knitr_1.48        htmltools_0.5.8.1
 [9] png_0.1-8         rmarkdown_2.27    cli_3.6.3         grid_4.4.1       
[13] withr_3.0.1       compiler_4.4.1    rstudioapi_0.16.0 tools_4.4.1      
[17] evaluate_0.24.0   Rcpp_1.0.13       yaml_2.3.10       rlang_1.1.4      
[21] jsonlite_1.8.8

Python

py_config()

python:         C:/Users/jjk1938/AppData/Local/miniforge3/envs/madpy/python.exe
libpython:      C:/Users/jjk1938/AppData/Local/miniforge3/envs/madpy/python312.dll
pythonhome:     C:/Users/jjk1938/AppData/Local/miniforge3/envs/madpy
version:        3.12.4 | packaged by conda-forge | (main, Jun 17 2024, 10:04:44) [MSC v.1940 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/jjk1938/AppData/Local/miniforge3/envs/madpy/Lib/site-packages/numpy
numpy_version:  2.0.1
os:             froze

NOTE: Python version was forced by use_python() function

Python

py_list_packages("madpy")

                     package            version
1                     brotli              1.1.0
2                 brotli-bin              1.1.0
3                      bzip2              1.0.8
4            ca-certificates           2024.7.4
5                      cairo             1.18.0
6                    certifi           2024.7.4
7                  contourpy              1.2.1
8                     cycler             0.12.1
9          double-conversion              3.3.0
10                     expat              2.6.2
11 font-ttf-dejavu-sans-mono               2.37
12      font-ttf-inconsolata              3.000
13  font-ttf-source-code-pro              2.038
14           font-ttf-ubuntu               0.83
15                fontconfig             2.14.2
16     fonts-conda-ecosystem                  1
17         fonts-conda-forge                  1
18                 fonttools             4.53.1
19                  freetype             2.12.1
20                 graphite2             1.3.13
21                  harfbuzz              9.0.0
22                       icu               75.1
23              intel-openmp           2024.2.0
24                kiwisolver              1.4.5
25                      krb5             1.21.3
26                     lcms2               2.16
27                      lerc              4.0.0
28                   libblas              3.9.0
29           libbrotlicommon              1.1.0
30              libbrotlidec              1.1.0
31              libbrotlienc              1.1.0
32                  libcblas              3.9.0
33                libclang13             18.1.8
34                libdeflate               1.21
35                  libexpat              2.6.2
36                    libffi              3.4.2
37                   libglib             2.80.3
38                  libhwloc             2.11.1
39                  libiconv               1.17
40                   libintl             0.22.5
41             libjpeg-turbo              3.0.0
42                 liblapack              3.9.0
43                    libpng             1.6.43
44                 libsqlite             3.46.0
45                   libtiff              4.6.0
46              libwebp-base              1.4.0
47                    libxcb               1.16
48                   libxml2             2.12.7
49                   libxslt             1.1.39
50                   libzlib              1.3.1
51     m2w64-gcc-libgfortran              5.3.0
52            m2w64-gcc-libs              5.3.0
53       m2w64-gcc-libs-core              5.3.0
54                 m2w64-gmp              6.1.0
55   m2w64-libwinpthread-git 5.0.0.4634.697f757
56                matplotlib              3.9.1
57           matplotlib-base              3.9.1
58                       mkl           2024.1.0
59         msys2-conda-epoch           20160418
60                   munkres              1.1.4
61                     numpy              2.0.1
62                  openjpeg              2.5.2
63                   openssl              3.3.1
64                 packaging               24.1
65                    pandas              2.2.2
66                     pcre2              10.44
67                    pillow             10.4.0
68                       pip               24.2
69                    pixman             0.43.4
70             pthread-stubs                0.4
71            pthreads-win32              2.9.1
72                 pyparsing              3.1.2
73                   pyside6              6.7.2
74                    python             3.12.4
75           python-dateutil              2.9.0
76             python-tzdata             2024.1
77                python_abi               3.12
78                      pytz             2024.1
79                     qhull             2020.2
80                  qt6-main              6.7.2
81                setuptools             72.1.0
82                       six             1.16.0
83                       tbb          2021.12.0
84                        tk             8.6.13
85                   tornado              6.4.1
86                    tzdata              2024a
87                      ucrt       10.0.22621.0
88                        vc               14.3
89              vc14_runtime        14.40.33810
90            vs2015_runtime        14.40.33810
91                     wheel             0.44.0
92               xorg-libxau             1.0.11
93             xorg-libxdmcp              1.1.3
94                        xz              5.2.6
95                      zlib              1.3.1
96                      zstd              1.5.6
                                  requirement     channel
1                                brotli=1.1.0 conda-forge
2                            brotli-bin=1.1.0 conda-forge
3                                 bzip2=1.0.8 conda-forge
4                    ca-certificates=2024.7.4 conda-forge
5                                cairo=1.18.0 conda-forge
6                            certifi=2024.7.4 conda-forge
7                             contourpy=1.2.1 conda-forge
8                               cycler=0.12.1 conda-forge
9                     double-conversion=3.3.0 conda-forge
10                                expat=2.6.2 conda-forge
11             font-ttf-dejavu-sans-mono=2.37 conda-forge
12                 font-ttf-inconsolata=3.000 conda-forge
13             font-ttf-source-code-pro=2.038 conda-forge
14                       font-ttf-ubuntu=0.83 conda-forge
15                          fontconfig=2.14.2 conda-forge
16                    fonts-conda-ecosystem=1 conda-forge
17                        fonts-conda-forge=1 conda-forge
18                           fonttools=4.53.1 conda-forge
19                            freetype=2.12.1 conda-forge
20                           graphite2=1.3.13 conda-forge
21                             harfbuzz=9.0.0 conda-forge
22                                   icu=75.1 conda-forge
23                      intel-openmp=2024.2.0 conda-forge
24                           kiwisolver=1.4.5 conda-forge
25                                krb5=1.21.3 conda-forge
26                                 lcms2=2.16 conda-forge
27                                 lerc=4.0.0 conda-forge
28                              libblas=3.9.0 conda-forge
29                      libbrotlicommon=1.1.0 conda-forge
30                         libbrotlidec=1.1.0 conda-forge
31                         libbrotlienc=1.1.0 conda-forge
32                             libcblas=3.9.0 conda-forge
33                          libclang13=18.1.8 conda-forge
34                            libdeflate=1.21 conda-forge
35                             libexpat=2.6.2 conda-forge
36                               libffi=3.4.2 conda-forge
37                             libglib=2.80.3 conda-forge
38                            libhwloc=2.11.1 conda-forge
39                              libiconv=1.17 conda-forge
40                             libintl=0.22.5 conda-forge
41                        libjpeg-turbo=3.0.0 conda-forge
42                            liblapack=3.9.0 conda-forge
43                              libpng=1.6.43 conda-forge
44                           libsqlite=3.46.0 conda-forge
45                              libtiff=4.6.0 conda-forge
46                         libwebp-base=1.4.0 conda-forge
47                                libxcb=1.16 conda-forge
48                             libxml2=2.12.7 conda-forge
49                             libxslt=1.1.39 conda-forge
50                              libzlib=1.3.1 conda-forge
51                m2w64-gcc-libgfortran=5.3.0 conda-forge
52                       m2w64-gcc-libs=5.3.0 conda-forge
53                  m2w64-gcc-libs-core=5.3.0 conda-forge
54                            m2w64-gmp=6.1.0 conda-forge
55 m2w64-libwinpthread-git=5.0.0.4634.697f757 conda-forge
56                           matplotlib=3.9.1 conda-forge
57                      matplotlib-base=3.9.1 conda-forge
58                               mkl=2024.1.0 conda-forge
59                 msys2-conda-epoch=20160418 conda-forge
60                              munkres=1.1.4 conda-forge
61                                numpy=2.0.1 conda-forge
62                             openjpeg=2.5.2 conda-forge
63                              openssl=3.3.1 conda-forge
64                             packaging=24.1 conda-forge
65                               pandas=2.2.2 conda-forge
66                                pcre2=10.44 conda-forge
67                              pillow=10.4.0 conda-forge
68                                   pip=24.2 conda-forge
69                              pixman=0.43.4 conda-forge
70                          pthread-stubs=0.4 conda-forge
71                       pthreads-win32=2.9.1 conda-forge
72                            pyparsing=3.1.2 conda-forge
73                              pyside6=6.7.2 conda-forge
74                              python=3.12.4 conda-forge
75                      python-dateutil=2.9.0 conda-forge
76                       python-tzdata=2024.1 conda-forge
77                            python_abi=3.12 conda-forge
78                                pytz=2024.1 conda-forge
79                               qhull=2020.2 conda-forge
80                             qt6-main=6.7.2 conda-forge
81                          setuptools=72.1.0 conda-forge
82                                 six=1.16.0 conda-forge
83                              tbb=2021.12.0 conda-forge
84                                  tk=8.6.13 conda-forge
85                              tornado=6.4.1 conda-forge
86                               tzdata=2024a conda-forge
87                          ucrt=10.0.22621.0 conda-forge
88                                    vc=14.3 conda-forge
89                   vc14_runtime=14.40.33810 conda-forge
90                 vs2015_runtime=14.40.33810 conda-forge
91                               wheel=0.44.0 conda-forge
92                         xorg-libxau=1.0.11 conda-forge
93                        xorg-libxdmcp=1.1.3 conda-forge
94                                   xz=5.2.6 conda-forge
95                                 zlib=1.3.1 conda-forge
96                                 zstd=1.5.6 conda-forge

Presentation repo: https://github.com/jjknack/reticulate-pres

Footnotes

https://lilicoding.github.io/papers/wang2020assessing.pdf
https://rstudio.github.io/reticulate/articles/python_packages.html
https://mamba.readthedocs.io/en/latest/index.html
https://rstudio.github.io/reticulate/
https://rstudio.github.io/reticulate/articles/calling_python.html
https://rstudio.github.io/reticulate/articles/rstudio_ide.html
https://rstudio.github.io/reticulate/articles/r_markdown.html
https://quarto.org/docs/computations/python.html
https://rstudio.github.io/reticulate/articles/calling_python.html#type-conversions
https://rstudio.github.io/reticulate/articles/arrays.html
https://rstudio.github.io/reticulate/articles/calling_python.html#sparse-matrices

Agenda

Agenda

About Me

Who is Jennifer Knack?

Quick aside: RStudio

Disclaimer

Don’t use Jupyter Notebooks.

RStudio is objectively the best IDE

“But why not just use VSCode?”

R syntax used in this presentation

Setup

Python install and package management

Setting up RStudio to run Python

Setting up RStudio to run Python

Setting up RStudio to run Python

reticulate

What does reticulate do?5

A note from Posit about the philosophy behind Python tools in Rstudio

What does reticulate do?

Set your environment

What does reticulate do?

The py object

The r object

Importing modules

Sourcing scripts

Executing code

Working in RMarkdown7

Working in Quarto

Working with Python interactively

What does reticulate do?

Type conversions

Conversion table

Controling when conversion happens

Defining the conversion

Indices

Arrays

Wait, do you mean vectors or matrices?

The TL;DR for arrays10

Column- vs. Row-Major Order

Reshaping arrays

Grouping while printing 3+D arrays

What about arrays from R to Python?

More array considerations

So in summary:

Sparse matrices

Data Frames

The important points

The important points

Using Pandas nullable data types

Final Points

For advanced Python users

Access to Python help from R

Some more reading

And I’m sure one of you will mention

Session info

R

Python

Python

Footnotes

What does reticulate do?⁵

The `py` object

The `r` object

Working in RMarkdown⁷

The TL;DR for arrays¹⁰